A human observer is equipped with the ability to assign colors to a given object under different illumination conditions. For example, most humans will assign a green color to grass at midday, when the main illumination is white sunlight, and also at sunset, when the main illumination is red. This ability of human perception system, which ensures that the perceived color of objects remains relatively constant under varying illuminations, may be referred to as color constancy. However, when grass images under different illuminations are viewed side-by-side, the difference in colors becomes more obvious to the human observer. When viewed in isolation without any background, the grass itself may not even appear green in color.
In digital image and video applications, computational color correction is sometimes used as an attempt to mimic the human ability to achieve color constancy. Color correction may comprise two steps: illumination estimation and chromatic adaptation. Illumination herein may refer to the application of a light source in an image or video scene. Illumination estimation may refer to the process of estimating the color, or spectral output, of the light source. This information may in turn be used to correct the color of the image or video to match a different illumination, the process of which may be referred to as chromatic adaptation.
Videoconferencing involving multiple users at remote locations is becoming increasingly popular. Telepresence is a term sometimes used to refer to video conferencing. In telepresence applications, a video object may refer to an area of interest in a video frame which comprises a human subject (foreground) separated from the surrounding environment (background). Multiple video objects from different sources may be transmitted to a common location in a telepresence system, where they may be received, processed and displayed together either with each video object in its individual background or with all video objects combined into a common background.
Telepresence users may likely have different illuminations at remote locations, unless special equipment such as controlled lighting rooms are used. In a telepresence system wherein each source video frame is complete and each video object may be perceived within its own background, the problem of perceived color differences between video objects may be somewhat minimal, since color constancy for a human observer is still preserved. However, the problem of perceived color differences may become significant in an immersive telepresence system that aims to create the virtual appearance of having all users situated in a same surrounding environment. In this paradigm, the original backgrounds from source video frames may be removed and video objects alone may be rendered onto a common background which may have an illumination different from each of the source video frames. Consequently, these video objects perceived side-by-side may not appear natural, as the color constancy model is no longer applicable.
To improve immersive telepresence, the color of multiple video objects in the same rendered frame may need to be consistent not only with each other but also with the common background, since brightness and/or color balance differences, however subtle, may be very noticeable. Color correction proves to be difficult in this case. Even if a white balance algorithm is implemented to correct the color of each entire video frame, this color correction may well not be the most appropriate color correction for each video object. Additionally, in an uncontrolled lighting environment—as may be found in a typical office or home, the color balance of a video object, as well as its perceived brightness, may change on a frame-by-frame basis. For example, movement by the video object in the foreground may occlude background light sources, and consequently change the illumination and/or the white balance of the video object. Thus, there remains a need to develop systems and/or methods of color correction to improve telepresence applications, such as immersive telepresence.